The use of synthetic images has become increasingly important and widespread in motion pictures and other commercial and scientific applications. A synthetic image represents a two-dimensional array of digital values, called picture elements or pixels, and thus can be regarded as a two-dimensional function. Image synthesis, then, is the process of creating synthetic images from scenes.
As a general matter, digital images are generated by rasterization (as described in greater detail below and in the references cited in this document, which are incorporated herein by reference as if set forth in their entireties herein), or, in the case of photorealistic images of three-dimensional scenes, by ray tracing (also as described in greater detail below and in the references cited herein). Both approaches aim at determining the appropriate color for each pixel by projecting the original function into the pixel basis. Due to the discrete representation of the original function, the problem of aliasing arises, as described below.
Image synthesis is perhaps the most visible part of computer graphics. On the one hand it is concerned with physically correct image synthesis, which intends to identify light paths that connect light sources and cameras and to sum up their contributions. On the other hand it also comprises non-photorealistic rendering, such as the simulation of pen strokes or watercolor.
The underlying mathematical task of image synthesis is to determine the intensity I (k, l, t, λ), where (k, l) is the location of a pixel on the display medium. Computing the intensity of a single pixel requires an integration function over the pixel area. This integral is often highly complex, as discussed below, and cannot be solved analytically, thus requiring numerical methods for solution, which may include Monte Carlo and quasi-Monte Carlo methods. In particular, image synthesis is an integro-approximation problem for which analytical solutions are available only in exceptional cases. Therefore numerical techniques need to be applied. While standard graphics text books still recommend elements of classical Monte Carlo integration, the majority of visual effects in movie industry are produced by using quasi-Monte Carlo techniques.
However, typical numerical methods used in such applications have their own limitations and attendant problems. It would therefore be desirable to provide improved methods and systems for image synthesis whereby realistic images can be rendered efficiently.
In computer graphics, a computer is used to generate digital data that represents the rejection of surfaces of objects in, for example, a three-dimensional scene illuminated by one or more light sources, onto a two-dimensional image plane, to simulate the recording of the scene by, for example, a camera. The camera may include a lens for projecting the image of the scene onto the image plane, or it may comprise a pinhole camera in which case no lens is used. The two-dimensional image is in the form of an array of picture elements, called “pixels” or “pels,” and the digital data generated for each pixel represents the color and luminance of the scene as projected onto the image plane at the point of the respective pixel in the image plane. The surfaces of the objects may have any of a number of characteristics, including shape, color, specularity, texture, and so forth, which are preferably rendered in the image as closely as possible, to provide a realistic-looking image.
Generally, the contributions of the light reflected from the various points in the scene to the pixel value representing the color and intensity of a particular pixel are expressed in the form of the one or more integrals of relatively complicated functions. Since the integrals used in computer graphics generally will not have a closed-form solution, numerical methods must be used to evaluate them and thereby generate the pixel value typically a conventional “Monte Carlo” method has been used in computer graphics to numerically evaluate the integrals. Generally, in the Monte Carlo method, to evaluate an integral
                              〈          f          〉                =                              ∫                                          [                                  0                  ,                  1                                )                            s                                ⁢                                    f              ⁡                              (                x                )                                      ⁢                          ⅆ              x                                                          (        1.1        )            where ƒ(x) is a real function on the s-dimensional unit cube [0,1)s, that is, an s-dimensional cube each of whose dimension includes “zero.” and excludes “one.” First, a number of N of statistically-independent randomly-positioned points x1, i=1, . . . , N, are generated over the integration domain. The random points xi are used as sample points for which sample values ƒ(xi) are generated for the function ƒ(x), and an estimate ƒ for the integral is generated as
                                          〈            f            〉                    ≈                      f            _                          =                              1            N                    ⁢                                    ∑                              i                =                1                            N                        ⁢                          f              ⁡                              (                                  x                  i                                )                                                                        (        1.2        )            
As the number of random points used in generating the sample points ƒ(xi) increases, the value of the estimate ƒ will converge toward the actual value of the integral (ƒ). Generally, the distribution of estimate values that will be generated for various values of N, that is, for various numbers of sample points, of being normal distributed around the actual value with a standard deviation σ which can be estimated by
                    σ        =                                            1                              N                -                1                                      ⁢                          (                                                                    f                    _                                    2                                -                                                      f                    _                                    2                                            )                                                          (        1.3        )            if the points xi used to generate the sample values ƒ(xi) are statistically independent, that is, if the points xi are truly positioned at random in the integration domain.
Generally, it has been believed that random methodologies like the Monte Carlo method are necessary to ensure that undesirable artifacts, such as Moiré patterns and aliasing and the like which are not in the scene, will not be generated in the generated image. However, several problems arise from use of the Monte Carlo method in computer graphics. First, since the sample points xi used in the Monte Carlo method are randomly distributed, they may clump in various regions over the domain over which the integral is to be evaluated. Accordingly, depending on the set of points that are generated, in the Monte Carlo method for significant portions of the domain there may be no sample points xi for which sample values ƒ(xi) are generated. In that case, the error can become quite large. In the context of generating a pixel value in computer graphics, the pixel value that is actually generated using the Monte Carlo method may not reflect some elements which might otherwise be reflected if the sample points xi were guaranteed to be more evenly distributed over the domain. This problem can the alleviated somewhat by dividing the domain into a plurality of sub-domains, but is generally difficult to determine a priori the number of sub-domains into which the domain should be divided, and, in addition, in a multi-dimensional integration region, which would actually be used in computer graphics rendering operations, the partitioning of the integration domain into sub-domains, which are preferably of equal size, can be quite complicated.
In addition, since the method makes use of random numbers, the error | ƒ−(ƒ)|, where |x| represents the absolute value of the value x between the estimate value ƒ and actual value (ƒ) is probabilistic, and, since the error values for various large values of N are close to normal distribution around the actual value (ƒ), only sixty-eight percent of the estimate values ƒ that might be generated are guaranteed to lie within one standard deviation of the actual value (ƒ).
Furthermore, as is clear from Equation (1.3), the standard deviation σ decreases with increasing numbers N of sample points, proportional to the reciprocal of square root of N, that is
      1          N        .Thus, if it is desired to reduce the statistical error by a factor of two, it will be necessary to increase the number of sample points N by a factor of four, which, in turn, increases the computational load that is required to generate the pixel values, for each of the numerous pixels in the image.
Additionally, since the Monte Carlo method requires random numbers to define the coordinates of respective sample points xi the integration domain, an efficient mechanism for generating random numbers is needed. Generally, digital computers are provided with so-called “random number generators,” which are computer programs which can be processed to generate a set of numbers that are approximately random. Since the random number generators use deterministic techniques, the numbers that are generated are not truly random. However, the property that subsequent random numbers from a random number generator are statistically independent should be maintained by deterministic implementations of pseudo-random numbers on a computer.
Grabenstein describes a computer graphics system and method for generating pixel values for pixels in an image using a strictly deterministic methodology for generating sample points, which avoids the above-described problems with the Monte Carlo method. The strictly deterministic methodology described in Grabenstein provides a low-discrepancy sample point sequence which ensures, a priori, that the sample points are generally more evenly distributed throughout the region over which the respective integrals are being evaluated. In one embodiment, the sample points that are used are based on the Halton sequence.
In a Halton sequence generated for number base b, where base b is a selected prime number, the k-th value of the sequence, represented by Hbk is generated by use of a “radical inverse” function Φb, that is generally defined as
                                                        Φ              b                        ⁢                          :                        ⁢                          N              0                                ->          l                ⁢                                  ⁢                  i          =                                                    ∑                                  j                  =                  0                                ∞                            ⁢                                                                    a                    j                                    ⁡                                      (                    i                    )                                                  ⁢                                  b                  i                                                      ↦                                          ∑                                  j                  =                  0                                ∞                            ⁢                                                                    a                    j                                    ⁡                                      (                    i                    )                                                  ⁢                                  b                                                            -                      j                                        -                    1                                                                                                          (        1.4        )            where (αj)j=0∞ is the representation of i in integer base b. Generally, a radical inverse of a value k is generated by technique including the following steps (1)-(3):
(1) writing the value k as a numerical representation of the value in the selected base b, thereby to provide a representation for the value as DMDM-1 . . . D2 D1, where Dm (m=1, 2, . . . , M) are the digits of the representation;
(2) putting a radix point, corresponding to a decimal point for numbers written in base ten, at the least significant end of the representation DMDM-1 . . . D2D1 written in step (1) above; and
(3) reflecting the digits around the radix point to provide 0. DMDM-1 . . . D2 D1, which corresponds to Hbk.
It will be appreciated that, regardless of the base b selected for the representation for any series of values one two, . . . k, written in base b, the least significant digits of the representation will change at a faster rate than the most significant digits. As a result in the Halton sequence Hb1, Hb2, . . . Hbk, the most significant digits will change at the faster rate so that the early values in the sequence will be generally widely distributed over the interval from zero to one, and later values in the sequence will fill in interstices among the earlier values in the sequence. Unlike the random or pseudo-random numbers used in the Monte Carlo method as described above, the values of the Halton sequence are not statistically independent; on the contrary, the values of the Halton sequence are strictly deterministic, “maximally avoiding” each other over the interval, and so they will not clump, whereas the random or pseudo-random numbers used in the Monte Carlo method may clump.
It will be appreciated that the Halton sequence as described above provides a sequence of values over the interval from zero to one, inclusive along a single dimension. A multi-dimensional Halton sequence can be generated in a similar manner, but using a different base for each dimension, where the bases are relatively prime.
A generalized Halton sequence, of which the Halton sequence described above is a special case, is generated as follows. For each starting point along the numerical interval from zero to one, inclusive, a different Halton sequence is generated. Defining the pseudo-sum x ⊕py for any x and y over the interval from zero to one, inclusive, for any integer p having a value greater than the pseudo-sum is formed by adding the digits representing x and y in reverse order, from the most significant digit to the least significant digit, and for each addition also adding in the carry generated from the sum of next more significant digits. Thus, if x in base b is represented by 0, XiX2 . . . XM−1XM, where each Xm is a digit in base b, and if y in base b is represented by 0. Y1Y2 . . . YN−1YN, where each Yn is a digit in base b, where M is the number of digits in the representation of x in base b, and where N is the number of digits in the representation of y in base b, and where M and N may differ, then the pseudo-sum z is represented by 0.Z1Z2 . . . ZL-1ZL, where each Z1 is a digit in base b given by Z1=(X1+Y1+C1) mod b, where mod represents the modulo function, and
      C    1    =      {                            1                                                                    for                ⁢                                                                  ⁢                                  X                                      t                    -                    1                                                              +                              Y                                  t                  -                  1                                            +                              Z                                  t                  -                  1                                                      ≥            b                                                0                          otherwise                    is a carry value from the 1-1-st digit position, with C1 being set to zero.
Using the pseudo-sum function as described above, the generalized Halton sequence that is used in the system described in Grabenstein is generated as follows. If b is an integer, and x0 is an arbitrary value on the interval from zero to one, inclusive then the p-adic von Neumann-Kakutani transformation Tb(x) is given by
                                          T            p                    ⁡                      (            x            )                          :=                  x          ⁢                      ⊕            p                    ⁢                      1            b                                              (        1.5        )            and the generalized Halton sequence x0, x1, x2, . . . is defined recursively asxn+1=Tb(xn)  (1.6)From Equations (1.5) and (1.6), it is clear that, for any value for b, the generalized Halton sequence can provide that a different sequence will be generated for each starting value of x, that is for each x0. It will be appreciated that the Halton sequence Hbk as described above is a special case of the generalized Halton sequence in Equations (1.5) and (1.6) for x0=0.
The use of a strictly deterministic low-discrepancy sequence such as the Halton sequence or the generalized Halton sequence can provide a number of advantages over the random or pseudo-random numbers that have are used in connection with the Monte Carlo technique. Unlike the random numbers used in connection with the Monte Carlo technique, the low discrepancy sequences ensure that the sample points are more evenly distributed over a respective region or time interval, thereby reducing error in the image which can result from clumping of such sample points which can occur in the Monte Carlo technique. That can facilitate the generation of images of improved quality when using the same number of sample points at the same computational cost as in the Monte Carlo technique.
It would also be desirable to provide methods and systems that provide image synthesis by adaptive quasi-Monte Carlo integration and adaptive integro-approximation in conjunction with techniques including a scrambled Halton Sequence, stratification by radical inversion, stratified samples from the Halton Sequence, deterministic scrambling, bias elimination by randomization, adaptive and deterministic anti-aliasing anti-aliasing rank-1 lattices, and trajectory splitting by dependent sampling and domain stratification induced by rank-1 lattices.